1. Field of the Invention
This invention relates to a solid-state imaging apparatus and, more particularly, to a single-chip color solid-state imaging apparatus which will be suitable for a novel television system for improving vertical resolution.
2. Description of the Prior Art
Interlaced scanning is adopted for the present standard television system, i.e. the NTSC system, of Japan and U.S.A. FIG. 1 of the accompanying drawings shows the principle of interlaced scanning. In the drawing, the ordinate represents the axis in the direction of vertical space in a television frame and the abscissa represents a time axis. One plane represents one field and symbol l does a scanning line. Each of the frames, which are transmitted and received 30 times per second, consists of two fields. Each field is scanned by scanning every other scanning line. Namely, odd-numbered scanning lines l.sub.11, l.sub.13, . . . , l.sub.1,263 are transmitted and received in the first field and even-numbered scanning lines l.sub.22, l.sub.24, . . . , l.sub.2,262 are transmitted and received in the second field. The first and second fields together constitute one frame. When this interlaced scanning is employed, a field frequency becomes 60 Hz even if a frame frequency is 30 Hz and this provides the effect as if the number of frames was 60 per second for human eyes, wherein flicker is eliminated. Since human eyes are characterized as having low-pass characteristics with respect to time, the number of scanning lines looks as if 525 scanning lines existed on a TV frame, though the number of scanning lines per 1/60 second is 262.5. Accordingly, the transmission band can be halved.
In order to accomplish interlaced scanning as described above, any of the following three signal readout methods shown in FIGS. 2(a) to 2(c) may be executed in a solid-state image sensing device. In the drawings, each square represents one pixel, the solid line represents the readout sequence of pixels of the first field and the broken line represents the read sequence of pixels of the second field. FIG. 2(a) shows frame storage one-row readout wherein alternate rows corresponding to each field are sequentially read out row-by-row. The light storage time of each pixel in such a read out method is one frame time wherein image lag develops. FIG. 2(b) shows field store two-row mixed readout wherein two vertical rows are mixed inside the image sensing device and the combination of the mixed two rows is alternately changed in each field. FIG. 2(c) shows field store two-row simultaneous readout wherein two vertical rows are not mixed but are simultaneously readout and their combination is changed in each field. In accordance with the signal readout methods shown in FIGS. 2(b) and 2(c) each pixel is readout once per 1/60 second.
A single-chip color solid-state imaging apparatus is one that obtains color television signals by use of one solid-state image sensing device executing any one of the signal readout methods described above. A filter having different spectral responsivity is disposed on the image sensing device for each pixel and signals in accordance with chrominance signals of a scene are sequentially outputted. A luminance signal providing data as to the brightness of the scene and a chrominance signal providing data on the color of the scene can be obtained by signal-processing this output. Various methods of disposing the color filters and various signal processing methods are available with respect to the three signal readout methods described above, as described, for example, in "IEEE Trans. Electron Devices", Vol. ED-32, pp. 1381-1389, Aug., 1985. As to the device for frame store one-row read out, refer to "Tech. Papers, Inst. Telev. Eng. Japan", Vol. 7, No. 41, pp. 1-6, (1984, 2). Refer also to "Tech. Papers, Inst. Telev. Eng. Japan", Vol. 8, No. 44, (1985, 2) pp. 1-6 for the device for field store two-row mixed readout, and "National Convention Rec. Inst. TV Eng. Japan", 4-13 (1980), pp. 83-84 as well as "Tech. Papers, Inst. Telev. Eng. Japan", Vol. 9, No. 45 (1986, 2) for the device for field store two-row simultaneous readout.
An example of a single-chip color solid-state imaging apparatus using a field store two-row simultaneous readout device will be described hereby with reference to FIGS. 3 and 4. FIG. 3 shows the circuit construction of a TSL solid-state image sensing device and the filter arrangement described in "National Convention Rec. Inst. TV Eng. Japan", 3-8 (1986), pp. 59-60. FIG. 4 is a block diagram of the signal processing system of the single-chip color solid-state imaging apparatus using the device shown in FIG. 3.
In FIG. 3, reference numeral 31 represents a photodiode for photoelectric conversion; 32 and 33 are vertical and horizontal shift registers for generating pulses for reading out sequentially signal charges of each photodiode, respectively; 34 is an interlace circuit for changing the combination of two rows that are to be readout simultaneously; 35 is a horizontal switch (or gate) that is opened and closed by the pulse of the horizontal shift register; 36 is a vertical switch (or gate) that is opened and closed by the pulse of the vertical shift register; 37 is a row switch (gate) for selecting the rows to be readout; 38 is a horizontal signal line; 39 is a vertical signal line; and 40 and 41 are reset drain and reset switch (or gate) for eliminating unnecessary charge in the horizontal signal line before signal readout. Symbols W, G, Cy, and Ye represent filters having white, green, cyan and yellow spectral responsivity disposed on each pixel, respectively.
Hereinafter, the operation of this device will be described. In the horizontal blanking period, the reset switch (gate) 41 opens, the unnecessary charges in all the horizontal signal lines are discharged to the reset drain 40 and then the reset switch (gate) 41 closes. Thereafter, the vertical switch (gate) 36 and the row switch (gate) 37 of two rows selected by the vertical shift register and the interlace circuit open. In this state, the horizontal shift register 33 opens and closes sequentially the horizontal switches and the signal charges of each pixel on which the .circle.W , .circle.G , .circle.Cy and .circle.Ye filters are disposed are outputted to the independent horizontal signal line 38 and vertical signal line 39.
In FIG. 4, reference numeral 42 represents the solid-state image sensing device shown in FIG. 3; 43 represents four preamplifiers for amplifying four output currents of the device; 44 represents a matrix circuit for providing the luminance signals y and the chrominance signals, r, g, b at the output thereof; 45 represents a process circuit for outputting the luminance signal y and color difference signals y-r, y-b from the y, r, g and b signals; and 46 represents an encoder for generating NTSC composite signals.
Hereinafter, the operation of this apparatus will be described. The chrominance signals W, Cy, Ye and G outputted from the solid-state image sensing device 42 are amplified by the corresponding four pre-amplifiers 43, respectively. The outputs of the amplifiers are respectively transmitted to the matrix circuit 44 which provides multiply/add signal processing and conversion resulting in the y, r, g or b signals in accordance with the following equation (1): ##EQU1##
After being subjected to white compression, gamma (.gamma.) correction, blanking addition, white balance and automatic gain control by the process circuit 45, the y, r, g and b signals are converted to the luminance signals y and the color difference signals y-r, y-b. Thereafter, each signal is subjected to quadrature modulation by the encoder 46 into the NTSC composite signal, that is, the output signal of this apparatus.
In the conventional apparatus described above, the luminance signals y and the color difference signals y-r, y-b of one scanning line are generated from the signals of two rows that are read out in one horizontal scanning period of the solid-state image sensing device, as can be seen clearly from the filter configuration shown in FIG. 3 and from the matrix circuit of the equation (1). In the solid-state image sensing devices for frame store one-row readout and for field store mixed readout, on the other hand, the color difference signals y-r, y-b are so-called line sequential signals wherein the y-r and y-b signals are alternately obtained by every horizontal scanning, but the luminance signal of one scanning line is generated from the signal or signals of one or two rows that are read out in one horizontal scanning period of the solid-state image sensing device.
Meanwhile, an EDTV system capable of transmission and reception of pictures having higher fineness by modifying the present television system has been proposed. An example of such an apparatus is disclosed in "J. Inst. Telev. Eng. Japan", Vol. 39, No. 10 (1985) pp. 891-897, Vol. 40, No. 3 (1986) pp. 154-161, "IEEE Trans. COM-32, 8, pp. 948-953 (Aug., 1984) and "SMPTE J.", 93, 10, pp. 923-929 (Oct., 1984). This system is directed primarily to improve effective vertical resolution by changing the scanning system of a television transmitter/receiver from interlaced scanning to sequential scanning. Hereinafter, the principle of improving vertical resolution by the change of scanning line disclosed in the above-mentioned literatures will be described with reference to FIGS. 5 to 8.
FIG. 5 shows a sampling frequency in the time-spatial frequency region of interlaced scanning and signal components of a picture when the image of a still picture is taken; FIG. 6 shows scanning line interpolation process for converting interlaced scanning, which is effected on the receiver side, to sequential scanning and characteristics of a scanning line interpolation filter; FIG. 7 is a block diagram of an EDTV color imaging apparatus for converting sequential scanning, which is effected on the transmission side, to interlaced scanning; and FIG. 8 is a block diagram of a motion-adaptive scanning converter and shows the characteristics of a pre-filter.
In interlaced scanning shown in FIG. 1, there is a time difference of 1/30 second from display of l.sub.22 to display of l.sub.42, for example, so that l.sub.31 and l.sub.33 displayed during this period interfere with each other visually, interline flicker develops and vertical resolution drops as a consequence. FIG. 5 illustrates this phenomenon in a sampling frequency region by regarding scanning as sampling in the time and spatial regions. In FIG. 5, the abscissa represents the vertical spatial frequency, the ordinate is the temporal frequency, black circles A-D are sampling frequency, the region S is the signal component of the picture when the image of a still picture is taken and S.sub.1 to S.sub.4 are sideband components generated by scanning.
Interline flicker is generated by the components of the region S.sub.4. The signal spectrum has expansion in the time-spatial frequency regions but in this case, too, the sideband components are generated around the point C, thereby causing disturbance.
Picture degradation can be removed by inserting a scanning line interpolation filter for removing the sideband components generated around the sampling frequency represented by the point C in FIG. 5 on the receiver side. FIG. 6 shows the principle of improving picture quality by this interpolation filter. FIG. 6(a) shows the scanning line structure of the receiver after insertion of this interpolation filter and its interpolation process. In the same way as in FIG. 1, the ordinate represents the axis in the vertical direction of the television frame and the abscissa represents the time axis. One plane represents one field and symbol l represents the scanning line. Symbol X represents the scanning lines that are not transmitted while the scanning lines with black point .cndot. represent the scanning lines that are transmitted. The broken line arrow represents the interpolation process at the still portion and one-dot-chain line represents the interpolation process at a moving picture portion. FIG. 6(b) shows the characteristics of such an interpolation filter. The abscissa represents the vertical spatial frequency, the ordinate represents the temporal frequency and black circles A to D correspond to the sampling frequency at the time of interlaced scanning. The hatched side is a passing area with symbol .circle.a representing the characteristics for the still picture .circle.b the characteristics for the moving picture and .circle.c the characteristics for the motion which is not so great.
Hereinafter, the interpolation process will be described. Insertion of the interpolation filter corresponds to conversion of interlaced scanning to sequential scanning as shown in FIG. 6(a), and data of the scanning lines X which are not transmitted are produced from the data of the scanning lines with black point .cndot. which are transmitted. This process is made by interpolation using the data of the previous field. In other words, as shown in FIG. 6(a), l.sub.32 is interpolated by l.sub.22 of the previous field, for example. The characteristics of the interpolation filter at this time correspond to symbol .circle.a in FIG. 6(b). However, this interpolation process involves the problems that interdigitated picture degradation occurs at a profile portion with respect to the motion in the horizontal direction and smoothness of motion is insufficient in the vertical direction. As to the moving pictures, therefore, interpolation is started from the upper and lower scanning lines in a given field. In other words, as shown in FIG. 6(a), l.sub.32 is interpolated by upper and lower scanning lines l.sub.31 and l.sub.33, for example. The characteristics of the interpolation filter at this time correspond to symbol .circle.b in FIG. 6(b). Furthermore, in order to make smooth switching between the still and moving pictures, the data of the previous field and the data of the upper and lower scanning lines in a given field are used in combination. The characteristics of the interpolation filter at this time correspond to symbol .circle.c in FIG. 6(b). The sideband components around the point C that cause interline flicker can be eliminated by inserting the interpolation filter described above, as shown in FIG. 6(b). Incidentally, though vertical resolution of the moving picture gets deteriorated according to this system, human eyes have characteristics such that resolution drops with respect to the moving picture. Therefore, this is not a critical problem at all.
Now, since the non-interlace arrangement is employed on the receiver side as described above, the problem of interline flicker can be solved, but the following two problems remain to be solved. First of all, in existing cameras for interlaced scanning, the width corresponding to two scanning lines in the frame is taken out in each field in order to prevent image lag. As a result, the interpolated scanning line from the previous field contains the same data half by half in the upper and lower scanning lines so that resolution characteristics in the vertical direction get deteriorated as much. Secondly, it is difficult to judge motion by the interlaced scanning signal by use of the existing cameras and hence the motion-adaptive processing described above cannot be effected so easily. To solve these two problems, there has been made a proposal according to which the camera, too, makes sequential scanning and this sequential scanning is converted to interlaced scanning. Hereinafter, a prior art example of the EDTV color imaging apparatus employing this system will be described. FIG. 7 is a block diagram of the EDTV color imaging apparatus and FIG. 8 shows a block diagram of its motion-adaptive scanning converter and the characteristics of a pre-filter.
In FIG. 7, reference numeral 71 represents a three-tube color camera for sequential scanning; 72 is an A/D converter for digital signal processing; 73 is a YIQ converter for converting the RGB signals to the luminance signal Y and the color signals I, Q (which are equal to R-Y and B-Y unless their bandwidth is dealt with s strictly); 74 is an aliasing detector circuit for detecting the motion; 75 is a motion-adaptive scanning converter for converting the YIQ signals from sequential scanning to interlaced scanning; 76 is an encoder for multiplexing the YIQ signals into the NTSC signals; and 77 is a D/A converter for converting the digital NTSC signals to analog signals.
FIG. 8(a) is a block diagram of the scanning converter, wherein reference numeral 74 represents an aliasing detector circuit having the characteristics such as shown in FIG. 8(b); 81 is a time-spatial filter exhibiting the characteristics (C-1) when there is no output of the aliasing detector circuit and the characteristics (C-2) when there is, as shown in FIG. 8(b); 82 is a time-spatial filter exhibiting the characteristics shown in FIG. 8(d); and 83 is a switch for 2:1 sub-sampling. FIGS. 8(b) to 8(d) each show the characteristics of the time-spatial filter. In the diagrams, the abscissa represents the vertical spatial frequency and the ordinate represents the temporal frequency. The hatched portion represents a passing band.
Hereinafter, the operation of this apparatus will be described. The optical signal is analyzed by a prism inside the three-tube color camera 71 and each of the R, G, B signals is subjected to photoelectric conversion of the camera tube and scanned sequentially by 525 scanning lines per 1/60 second to provide each output. After which each of the R, G, B signals is converted to a digital signal by the A/D convertor 72 at a speed which is eight times that of the color sub-carrier and turned to the luminance signal Y and the chrominance signals I, Q by the YIQ converter. The YIQ signal is thinned out per each scanning line by the motion-adaptive scanning converter 75 and converted to interlaced scanning. In order to avoid aliasing distortion due to sub-sampling in this instance, each signal passes through the pre-filter having the characteristics as shown in FIG. 8. In other words, when no output of the aliasing detector circuit 74 exists (corresponding to the still picture) band limitation is not applied to the luminance signal and when this output exists (corresponding to the moving picture), band limitation shown in FIG. 8 C-2 is effected. Accordingly, the drop of vertical resolution attributable to the characteristics of time-spatial filter is eliminated in the still picture and resolution can be improved. On the other hand, since the resolution characteristics of the visual sense to the color difference signals are much lower than those of the luminance signal, no aliasing distortion occurs in the chrominance signal under the all-time band limitation of FIG. 8(d). The signal converted to interlaced scanning is turned to the NTSC signal by the encoder 76 and thereafter to the output signal through the D/A converter 77.
According to the prior art example described above, the data of the scanning line interpolated between the fields on the receiver side do not at all have the same portion as the data of the scanning lines of the present field in the case of the still picture, and resolution of 525 scanning lines, as such, can be accomplished. The sampling frequency of the point C in FIG. 5 is eliminated by sequential scanning and motion can be detected easily by the time-spatial filter.
As can be understood from the operation of the adaptive type scanning converter, it is only the luminance signal that needs 525 scanning lines per 1/60 second in order to accomplish the EDTV color imaging apparatus and the chrominance signal components need only 525/2 scanning lines for the chrominance signal is not used for the purpose of motion detection and band limitation of 525/4 lines is always effected.
The conventional single-chip color solid-state imaging apparatus described above does not take sequential scanning, necessary for the EDTV system, into consideration and hence involves the problem that the number of pixels in the vertical direction of the solid-state image sensing device must be doubled in order to accomplish the EDTV single-chip color imaging apparatus by the conventional technique for the number of scanning lines per field in sequential scanning is twice the number of scanning lines per field in interlaced scanning. The increase in the number of pixels in the vertical direction causes a high level of integrated circuit technology to be needed for fabricating the device and S/N (signal to noise ratio) drops due to the drop of the fill factor (or the ratio of photosensitive area) of the device. For these reasons, it has been extremely difficult to accomplish the EDTV single-chip color device.